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1.  Introduction 


Volume  visualization  is  a  technique  used  in  scientific  visualization  and  computer-based  imaging 
to  represent  an  entire  volume  of  data  in  such  a  way  as  to  allow  for  complex  physical  phenomena 
to  be  clearly  displayed.  The  resulting  image  can  best  be  described  as  an  opaque  cloud  of  data 
that  utilizes  a  color  scale  to  represent  values  within  the  computational  mesh.  The  ability  to 
manipulate  and  animate  this  volume  “cloud”  over  time  provides  insight  into  a  large  three- 
dimensional  dataset  and  allows  the  viewer  to  see  the  results  of  millions  (or  billions)  of 
computational  values  simultaneously.  Volume  visualization  allows  the  analyst  to  view  an  entire 
range  of  values  within  a  computational  domain  rather  than  calculate  a  specific  scalar  value  for  an 
isosurface  or  a  range  of  values  on  a  simple  two-dimensional  clip  plane.  By  adjusting  minimum 
and  maximum  data  ranges  and  through  the  judicious  use  of  color  opacity,  analysts  can  make 
complex  physical  phenomena  clearly  visible  within  the  rendered  volume. 

There  are  many  algorithms  available  to  perform  volume  rendering,  including  volume  ray  casting, 
splatting,  shear  warp,  texture  mapping,  and  various  hardware-accelerated  rendering  techniques. 

It  is  not  the  purpose  of  this  report  to  describe  or  evaluate  these  various  techniques,  as  they  are 
well  documented  in  numerous  published  papers  (Drebin  et  al.,  1988;  Howison  et  al.,  2012; 
Kaufman,  1991;  Wittenbrink,  1998).  However,  the  availability  of  the  current  generation  of 
high-performance  parallel  computing  systems  and  fast  graphics  processors  allows  these  various 
techniques  to  excel  in  a  variety  of  implementations. 

Volume  visualization  concepts  have  existed  since  the  1970s  and  emerged  in  the  1990s  as  a  key 
field  of  visualization,  computer  graphics,  and  computer  imaging  (Kaufman,  1991).  Volume 
rendering  was  applied  initially  to  medical  imaging,  and  most  of  the  early  implementations  were 
application  specific,  not  general  purpose  in  nature.  Since  that  time,  the  algorithms  have  evolved 
to  be  more  general  purpose,  and  mature  open-source  visualization  applications,  such  as 
ParaView  and  Visit,  have  included  volume-rendering  capabilities  for  many  years,  implementing 
techniques  and  algorithms  developed  primarily  for  the  large  Department  of  Energy  research 
laboratories.  However,  initial  efforts  to  use  these  open-source  general-purpose  applications  for 
volume  visualization  of  data  typically  generated  at  the  U.S.  Army  Research  Laboratory  (ARL) 
was  limited  either  by  file  format,  data  size,  or  computational  resources,  and  the  results  were 
marginally  successful  at  best. 
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2.  Methodology 


Since  the  early  1990s,  ARL  has  been  using  the  commercial  software  package  EnSight  from 
Computational  Engineering  International  (CEI)  as  a  general-purpose  visualization  package. 
Computational  scientists  within  ARL  currently  use  EnSight,  in  conjunction  with  ParaView  and 
Visit,  for  high-performance  scientific  visualization.  EnSight  has  been  a  valuable  tool  for  post¬ 
processing  computational  results  and  has  been  effective  at  providing  general-purpose 
visualization  across  a  number  of  computational  disciplines  at  ARL.  Most  of  the  visualization 
techniques  in  EnSight  use  points,  lines,  and  surfaces  to  represent  data,  requiring  volumetric  data 
to  be  subsampled  in  some  way  (EnSight,  2012).  Traditionally,  analysts  would  rely  on  clip 
planes,  contour  plots,  isosurfaces,  isovolumes,  and  particle  traces  (figure  1)  to  extract 
information  from  their  computational  mesh.  These  techniques  allow  analysts  to  view  a  specific 
value  or  small  range  of  values  within  their  dataset  and,  in  most  cases,  animate  those  results  over 
time.  Certain  aspects  of  the  calculation  are  visible  using  these  techniques,  and  the  results  are 
effective  for  a  concise  range  of  values. 

Volume  rendering  was  first  introduced  in  EnSight  in  2010,  and  the  initial  implementation  was  a 
computationally  expensive  process.  The  ability  to  generate  volumes  was  tied  directly  to  the 
performance  and  memory  of  a  local  graphics  video  card.  This  hardware-accelerated  unstructured 
volume  Tenderer  used  a  hybrid  raycasting  system  that  decomposed  the  geometry  into  tetrahedra 
and  pushed  that  representation  directly  to  the  graphics  card  (Frank,  201 1).  Performance  of  this 
algorithm  varied  directly  with  the  size  of  the  computational  mesh,  the  size  of  the  image  being 
rendered,  and  the  complexity  of  the  scene,  but  the  overriding  implementation  issue  was  the 
amount  of  memory  required  on  the  local  graphics  card  to  render  the  image.  Because  of  the  size 
of  datasets  typically  generated  at  ARL,  it  turned  out  that  this  method  was  not  practical  for  any 
real-world  datasets,  including  computational  results  typically  addressed  at  ARL. 

In  2012,  EnSight  10.0  introduced  a  new  volume-rendering  technique.  This  updated  technique 
maintains  the  original  technique  that  was  dependent  on  exclusive  use  of  the  graphics  card  but 
with  a  significantly  improved  algorithm.  However,  this  method  continued  to  be  too  limiting  for 
general-purpose  use.  Fortunately,  EnSight  implemented  a  second  algorithm  that  uses  a 
structured  grid  to  remap  the  computational  mesh.  This  second  method  allows  the  end  user  to 
resample  the  data  to  any  resolution  by  creating  an  underlying  structured  volume  based  on  values 
provided  by  the  analyst.  Based  on  positive  performance  results  obtained  on  initial  testing  of 
large  datasets  generated  at  ARL,  this  methodology  is  very  efficient,  and  EnSight  is  able  to  better 
leverage  the  computational  resources  with  a  structured  volume  algorithm.  The  volume 
calculated  is  less  precise  than  the  original  hardware-based  rendering  methodology  but  faster  and 
more  flexible  in  terms  of  end  user  interactive  response.  Despite  the  reduced  accuracy  of  the 
calculated  volume,  there  is  often  little  to  no  perceivable  difference  in  the  rendering  results. 
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Figure  1.  Traditional  visualization  techniques:  (a)  particle  trace,  (b)  cutting  plane,  (c)  isosurface,  and  (d) 
clip  plane  and  isosurface  applied  to  ducted  rotor  dataset. 
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This  new  methodology  also  allows  the  analyst  to  subset  the  dataset  using  a  bounding  box, 
therefore  focusing  the  volume -rendering  algorithm  to  a  specific  area  of  interest.  This 
methodology  is  important  from  a  computational  standpoint — the  volume-rendering  algorithms 
are  computationally  expensive,  as  they  must  look  at  every  cell  within  the  mesh  multiple  times. 

By  subsetting  the  dataset,  analysts  are  able  to  significantly  reduce  the  amount  of  computational 
effort  required  to  calculate  the  volume  rendering.  Additionally,  when  a  structured  grid  is 
resampled  from  unstructured  data,  the  sample  is  focused  on  the  region  of  interest  rather  than  the 
entire  computational  mesh,  therefore  reducing  the  computational  effort  by  reducing  the  number 
of  cells  included  in  the  volume  calculation.  Subsetting  the  volume  is  limited  by  a  specific 
rectangular  area  oriented  in  any  direction  and  by  identifying  a  specific  variable  to  be  volume 
rendered.  EnSight  will  volume  render  that  variable  within  the  defined  bounding  box  area 
(Grimsrud,  2012). 

Volume  rendering  was  successfully  tested  using  both  a  traditional  client-server  connection 
between  the  desktop  workstation  and  the  ARL  utility  server  as  well  as  the  HPCMP-provided 
secure  remote  desktop  (SRD)  application.  When  a  traditional  client-server  connection  is  used 
with  EnSight,  the  volume  Tenderer  depends  on  the  local  graphics  hardware  on  the  desktop 
workstation  to  display  the  volume.  When  volume  animations  are  created  in  batch,  the  EnSight 
volume  Tenderer  requires  the  use  of  the  ARL  utility  server,  specifically  the  back-end  graphics 
nodes,  as  the  rendering  algorithm  still  relies  on  the  availability  of  graphics  hardware.  (The 
volume  Tenderer  cannot  be  used  on  traditional  compute  nodes  that  do  not  have  graphics 
hardware.)  A  sample  EnSight  command  file  and  Portable  Batch  System  (PBS)  batch  script  used 
for  creating  a  volume  animation  are  included  in  the  appendix  of  this  report. 

From  an  end  user  perspective,  during  early  exploration  of  the  dataset,  a  very  low-resolution  (such 
as  32  x  32  x  32)  structured  mesh  can  be  used  to  get  general  characteristics  of  the  data  in  the 
selected  region.  As  the  analyst  begins  to  manipulate  the  color  palette  to  extract  features  of 
interest,  the  resolution  of  the  structured  mesh  can  be  increased  to  show  more  detail.  Previewing 
the  volume  using  lower-resolution  structured  mesh  will  show  individual  planes  used  in  the 
volume -rendering  technique.  As  the  resolution  of  the  structured  mesh  is  increased,  the  details  of 
the  mesh  become  visually  insignificant.  The  results  of  changing  the  resolution  of  the  structured 
mesh  are  demonstrated  in  figure  2.  The  lower-resolution  images  visibly  demonstrate  how  the 
data  has  been  subsampled  using  a  structured  mesh.  As  the  resolution  of  the  structured  mesh 
increases,  the  result  volume  rendering  becomes  more  precise,  features  become  more  sharply 
defined,  and  the  visual  artifacts  of  the  subsampling  disappear. 

Early  testing  of  the  algorithm  indicates  that  a  final  resolution  of  200  for  the  subsampling  mesh 
seems  to  be  sufficient  for  creating  video  animations,  and  a  resolution  of  500  provides  high 
enough  resolution  for  printed  renderings  of  the  dataset.  These  values  may  ultimately  be  dataset 
dependent,  and  additional  use  of  this  algorithm  will  further  refine  these  “rules  of  thumb”  values. 
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(e)  250  x  250  x  250  structured  mesh _ (f)  500  x  500  x  500  structured  mesh 


Figure  2.  Result  of  changing  the  resolution  of  the  resampled  volume  data  using  a  structured  mesh. 
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This  volume-rendering  technique  allows  the  analyst  to  view  an  entire  volume  of  data  at  once, 
without  having  to  slice  the  data  in  any  way.  The  process  involves  finding  all  cells  that  overlap 
each  pixel  and  accumulating  a  color  for  that  pixel  from  the  color  and  transparency  of  each  cell. 
Volumes  can  be  constant  colored,  but  more  often  the  analyst  will  color  the  volume  using  a 
palette  corresponding  to  a  variable  or  a  variable  component.  Color  palettes  include  transparency 
(“alpha”)  values,  allowing  control  of  transparency  based  on  variable  value  (EnSight,  2012). 


3.  Case  Study 


The  ARL  Data  Analysis  and  Assessment  Center  (DAAC)  team,  part  of  the  ARL  Department  of 
Defense  Supercomputing  Resource  Center  (DSRC),  took  this  opportunity  to  demonstrate  the  new 
volume -rendering  techniques  made  available  in  EnSight  10.0  to  the  research  community  at  ARL. 
Because  of  the  complexity  and  limitations  experienced  with  previous  volume  visualization 
implementations,  the  customer  base  at  ARL  had  not  been  exposed  to  this  technique.  ARL 
visualization  experts  had  limited  experience  and  success  with  previous  volume -rendering  efforts 
and  wanted  to  explore  the  value  of  EnSight’ s  implementation  as  applied  to  real-world  datasets. 

In  an  effort  to  demonstrate  the  use  of  volume  rendering  to  the  ARL  EnSight  user  community,  an 
example  of  volume  rendering  was  developed  and  compared  to  traditional  isosurface  rendering 
techniques  and  presented  to  potential  users  (figure  3).  This  sample  volume  rendering  generated 
interest  from  the  user  community,  and  several  datasets  were  evaluated  for  potential  application  of 
volume  rendering. 

Of  particular  interest  was  a  computational  fluid  dynamics  dataset  of  a  ducted  rotor.  This  time- 
dependent  dataset  demonstrates  how  volume  rendering  can  show  the  dynamic  physics  associated 
with  a  spinning  vehicle  rotor.  Specifically,  this  simulation  represents  a  ducted  rotor  in  hover, 
and  these  rotors  would  be  used  for  small  unmanned  aerial  vehicles.  This  particular  simulation  is 
being  used  to  study  the  effect  of  tip  clearance  on  the  performance  of  the  rotor  in  hover.  The 
volume  visualization  clearly  shows  the  transient  flow  field  as  the  rotor  starts  spinning  and  how 
the  quiescent  fluid  is  disturbed  as  the  rotor  spins  up  and  reaches  a  quasi-steady  (periodic)  state 
(Dinavahi,  2012).  This  dataset  allowed  the  EnSight  volume -rendering  features  to  be  fully 
exercised  and  used  most  of  the  features  required  to  create  a  successful  volume: 

•  Subsetting  the  data  using  the  box  tool  to  highlight  a  specific  region  within  the 
computational  mesh. 

•  Resampling  an  unstructured  dataset  using  a  three-dimensional  structured  mesh  within  the 
region  of  interest. 

•  Judicious  editing  of  the  opacity/intensity  levels  of  the  color  palette  to  extract  detail  from 
the  volume  rendering. 
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Figure  3.  Comparing  traditional  visualization  results  to  volume -rendered  results:  (a-c)  shows  a 

traditional  isosurface  visualization  of  a  time-dependent  dataset,  and  (d)  demonstrates  how 
volume  visualization  can  be  used  to  depict  the  data  in  such  a  way  as  to  see  the  entire  volume, 
not  just  the  surface  data. 


7 


Displaying  a  part  or  parts  in  EnSight  as  a  volume -rendered  object  is  accomplished  via  EnSight’s 
“part  element  representations”  feature.  Parts  are  generally  loaded  into  EnSight  by  default  in 
Border,  Feature  Angle,  or  Full  representation.  These  representations  include  only  point,  line,  or 
surface  data.  The  “volume”  element  representation  will  activate  volume  rendering  for  a  part. 

Just  as  for  surfaces,  the  part  color  dialog  can  be  used  to  control  the  color  and  opacity  of  the 
volume.  The  volume  can  be  colored  by  a  scalar,  vector,  or  vector  component,  and  the  colors  will 
be  assigned  based  on  the  palette  (EnSight,  2012).  Figure  4  shows  how  to  represent  an  EnSight 
part  as  a  volume  through  the  EnSight  graphical  user  interface  (GUI). 

The  method  for  refining  the  results  from  the  volume  rendering  is  described  in  this  next  critical 
step.  EnSight  allows  for  advanced  control  of  volume  rendering  by  providing  a  mechanism  to 
adjust  the  magnitude  of  the  variable  palette  and  by  adjusting  the  alpha  value.  The  alpha  value  is 
the  opacity/intensity  of  the  particular  computational  variable  when  volume  rendered.  This  means 
that  variable  levels  within  the  volume  rendering  can  be  emphasized  by  specifically  increasing  the 
alpha  magnitude  of  these  select  levels  (EnSight,  2012).  Conversely,  specific  values  can  be 
removed  from  the  volume  by  adjusting  the  alpha  magnitude  to  zero.  For  instance,  there  may  be  a 
variable  that  represents  a  common  value  such  as  “air”  that  the  analyst  wants  to  remove  from  the 
volume.  By  selecting  the  range  of  scalar  values  that  represent  air  and  setting  the  opacity  to  zero, 
the  analyst  removes  the  “air”  from  the  volume,  allowing  the  volume  Tenderer  to  focus  on  other 
critical  values. 

The  analyst  can  also  scale  up  the  entire  alpha  by  adjusting  a  multiplier  value  in  the  palette 
editor’s  “Options”  tab.  The  analyst  can  also  achieve  sharp  delineations  in  the  rendered  volume 
by  adjusting  the  variable  palette  using  a  “sawtooth”  pattern.  Creating  very  abrupt  changes  in  the 
opacity/intensity  value  at  the  correct  value  within  the  scalar  variable  range  will  result  in  a  sharp 
peak  in  the  representation  of  that  data  point  in  the  volume  image.  The  default  color  palette  does 
not  highlight  the  detail  available  within  the  volume.  By  carefully  selecting  values  with  the  scalar 
range  to  highlight,  the  analyst  can  visually  emphasize  critical  elements  of  the  calculation.  This 
technique  is  represented  in  figure  5.  In  figure  5a,  a  bounding  box  is  created  to  subsample  the 
computation  mesh  to  the  area  of  interest.  The  volume -rendered  results  are  then  displayed  (5b) 
based  on  the  default  color  palette  values.  Figure  5c  shows  the  color  palette  editors  with  default 
values  for  opacity/intensity  and  the  min/max  variable  values,  and  5d  displays  the  adjusted  palette 
with  the  min/max  range  of  the  UVW  variable  adjusted  and  the  opacity/intensity  values  adjusted. 
The  final  image  in  figure  5e  emphasizes  the  complex  fluid  disturbances. 
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(a)  From  the  feature  bar  of  the  main  window  of  the  EnSight  GUI,  select  the  parent  part  (the  part 
that  is  to  be  displayed  as  a  volume),  and  using  the  “part  element  representation”  icon,  change  the 
object  to  be  displayed  as  a  volume. 


Structured  volume  rendering  suggestion 


Volume  representation  operation  is  estimated  to  use 
79MBs.  Consider  using  volume  rendering  with  a 
structured  box  clip  instead. 

<§>  Use  structured  box  clip  O  Use  volume  rep.  as  selected 


-Structured  dimensions-! 

X 
Y 
Z 


16 

▲ 

▼ 

16 

▲ 

-W 

16 

▲ 

▼ 

-Color  volume  by  variable 


□  Show  palette  tool  on  okay 


□  Don't  show  this  message  again  for  this  session. 


0  Ok 


X  Cancel 


(b)  After  changing  the  element  representation  to  be  “volume”,  use  a  structured  clip  box  to  create 
the  volume,  define  the  size  of  the  three-dimensional  structured  clip  used  to  subsample  the 
dataset,  and  select  the  variable  to  be  used  to  generate  the  volume. 


Figure  4.  How  to  display  an  EnSight  part  as  a  volume  via  the  GUI. 
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1-' 

uvw 

1.572e+02  i— | 
1.1796+02  1 
7.860e+01  1 
3.9306+01  1 
0.000e+00  «-J 

(a) 


(b) 


(e) 


Figure  5.  Using  the  EnSight  palette  editor  to  extract  details  from  the  volume. 


10 


4.  Conclusion 


Volume  visualization  provides  additional  insight  into  the  analysis  of  large  three-dimensional 
datasets.  Traditional  visualization  techniques,  such  as  clip  planes  and  isosurfaces,  depend  on 
looking  at  a  subset  of  the  calculated  data  and  representing  those  results  on  the  surface  of  the 
geometric  object.  Volume  rendering  allows  millions  (or  billions)  of  calculated  values  to  be 
displayed  simultaneously.  Through  judicious  adjustments  to  the  color  palette  opacity  and 
intensity  levels,  enormous  amounts  of  information  can  be  extracted  for  an  entire  volume  of  data. 
The  volume -rendering  technique  implemented  in  EnSight  provides  the  end  user  with  an  easy-to- 
use  interface  and  an  efficient  algorithm  to  either  quickly  display  low-resolution  representations 
of  the  volume  or  generate  very  high -resolution  (yet  computationally  expensive)  volume  images. 
By  using  a  bounding  box  to  select  the  region  of  interest  within  the  computational  mesh,  the 
analyst  can  focus  the  volume-rendering  algorithm  on  a  specific  portion  of  the  mesh,  therefore 
saving  valuable  computational  time.  The  use  of  a  structured  mesh  to  subset  the  data  provides  a 
reasonable  approximation  of  the  values  within  the  domain,  and  the  ability  to  manipulate  the 
resolution  of  the  resampled  data  allows  the  analyst  to  refine  the  rendering  to  extract  valuable 
features  from  the  data  that  would  not  have  previously  been  available  using  traditional  rendering 
techniques.  Real-time  volume  rendering  has  matured  to  the  point  where  it  can  become  a 
powerful  tool  in  the  ARL  researcher’s  interactive  analytical  toolkit. 
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Appendix.  Sample  EnSight  Batch  Command  File  and  Portable  Batch  System 
(PBS)  Script 


The  EnSight  Batch  Command  File  “sample. enc”  was  developed  for  the  Department  of 
Defense/High-Performance  Computing  Modernization  Program  utility  server  located  at  the  U.S. 
Army  Research  Laboratory. 


VERSION  10.02 

#  Restore  EnSight  context  file 

file:  restore_context  Vorticity_Magnitude.ctx 

# 

#  Select  the  volume  part  and  change  density  of  structured  volume  mesh  (2003) 
part:  select_begin 

19 

part:  select_end 

part:  modify_begin 

clip:  sample_step  200  200  200 

part:  modify_end 

# 

anim_recorders:  render_off screen  ON 
file:  image_numpasses  4 
file:  image_stereo  current 
file:  image_screenjiling  1  1 

#  set  output  file  name 

file:  animation_file  vortmag_notext 

#  set  output  image  dimensions 

file:  animation_window_size  user_defined 
file:  animation_window_xy  1600  1200 

file:  animation_frames  170 
file:  animation_multiple_images  OFF 
file:  animation_play_flipbook  OFF 
file:  an i mation_play_time  ON 
file:  animation_play_keyframe  OFF 
file:  animation_reset_flipbook  OFF 
file:  an imation__resct_t races  OFF 
file:  animation_reset_time  ON 
file:  animation_reset_keyframe  OFF 
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file:  save_animation 


PBS  Batch  Script 

#!/bin/csh  -f 

#PBS  -A  ARLAP 123456 
#PBS  -N  VortBatch 
#PBS  -q  vnc 

#PBS  -1  walltime=3 6 : 00 : 00 

#PBS  -1  select=l:mpiprocs=16:ncpus=16:ngpus=l 

#PBS  -1  place=scatter:excl 

#PBS  -1  application=ensight 

#  initialize  modules  environment  and  load  default  ensight  module 

source  /app/modules/init/csh 
module  load  ensight 
module  list 

cd  -angel/Surya 

setenv  ENSIGHT  10_MAX_CTHREADS  16 
setenv  EN  S IGHT 1 0_MAX_S  OS  THREAD  S  16 
setenv  ENS  IGHT  10_MAX_THRE ADS  16 

echo  HOSTNAME=  'hostname  -s' 

#  Set  up  working  directory  ${ CENTER} /batch 

if  (!  -d  $ { CENTER } /batch)  mkdir  ${ CENTER} /batch 
cp  VortMag2/VortMag_NoText.ctx*  ${ CENTER} /batch 
cp  VortMag2/notext.enc  ${ CENTER} /batch 
cd  ${ CENTER} /batch 

#  Run  Ensight,  using  the  notext. enc  EnSight  command  file  as  input 

env  DISPLAY=:0.0  /app/VirtualGL/bin/vglrun  ensightlOO  -batch  -gold  -no_start_screen  \ 
-c  localhost  -p  sample. enc 
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